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The  National  Defense  Authorization  Act  for  Fiscal  Year  1996  (Public 
I^w  104-106)  directed  us  to  review  the  Department  of  Defense  (dod)  Joint 
Logistics  Systems  Center’s  (jlsc)  development  and  deployment  of  standard 
materiel  management  systems.  These  systems  are  being  developed  in 
order  to  modernize  and  improve  materiel  management  business 
operations  and  processes.  The  effort  involved  developing  a  standard  suite 
of  rune  separate  materiel  management  systems,  or  applications,  to  form 
the  Materiel  Management  Standard  System  (mmss).  The  program  is  also 

part  of  Defense  s  Corporate  Information  Management  (cim)  initiative _ a 

departmentwide  effort  to  improve  operations  and  reduce  costs  by 
streamlining  business  processes,  consolidating  information  systems  and 
standardizmg  and  integrating  data.  As  of  July  1995,  Defense  estimated  that 
it  would  sperid  about  $6.3  billion  to  develop,  deploy,  and  maintain  the 
system  at  17  inventory  control  points,  and  it  expected  the  effort  to 
produce  as  much  as  $15  billion  in  savings  over  a  16-year  period.  At  the 
close  of  fiscal  year  1995,  Defense  had  spent  a  reported  $714  million  on 
development.^ 

course  of  our  review,  however.  Defense  decided  to  undertake  a 
afferent  approach  to  developing  materiel  management  systems  because  of 
tundmg  cute,  cost  overruns,  and  schedule  delays.  Also,  individual  services 
were  pressing  for  quicker  system  deployment.  Under  the  new  approach, 
the  materiel  management  systems  will  not  be  standard  or  integrated. 
Instead,  each  of  the  nine  system  applications  vidll  be  individually  and 
incrementally  developed  and  deployed  at  selected  inventory  control  points 
between  fiscal  years  1996  and  1999.  The  military  services  and  the  Defense 
Logistics  Agency  (dla)  will  choose  which  applications  they  want,  and 
soirie  inventory  control  points  may  never  receive  new  systems. 

Deployment  wiU  be  constrained  by  available  funding.  This  is  a  m^or 
departure  from  dod’s  previous  goal  of  eliminating  multiple  and  redimdant 
business  processes  and  hundreds  of  legacy  (current)  systems  and  moving 
to  a  standard,  corporate  dod  logistics  process  and  system. 

Because  of  this  change,  we  focused  our  review  primarily  on  Defense’s  new 
strategy  and  the  extent  to  which  it  will  facilitate  improvement  of  materiel 
management  business  processes  and  reduce  costs  associated  with 
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I’me  figure  does  not  include  a  reported  $246  million  spent  on  developing  and  implementing  near  term 
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Results  in  Brief 


inventory,  personnel,  and  inefficient  legacy  systems.  This  report 
summarizes  some  of  the  problems  Defense  experienced  with  the  former 
strategy  and  discusses  initial  concerns  we  have  with  the  recent  change  in 
Defense’s  approach. 


In  December  1995,  Defense  determined  that  its  goal  of  developing  a 
standard  suite  of  nine  integrated  systems  to  improve  various  aspects  of 
materiel  management  operations — ^including  asset  management, 
requirements  determination,  and  inventory  management— would  cost 
much  more  than  the  $5.3  billion  originally  estimated.  Defense  abandoned 
its  plan  to  deploy  all  nine  systems  as  an  integrated  suite  across  all 
inventory  control  points  and  now  plans  to  deploy  each  system  individually 
as  it  is  developed  at  a  selected  site.  It  has  also  embarked  on  an  accelerated 
deployment  schedule  to  provide  these  systems  over  the  fiscal  year  1996 
through  1999  time  period. 

Defense  is  embarking  on  a  new  strategy  before  taking  a  number  of  steps 
needed  to  ensure  that  the  additional  planned  hundreds  of  millions  of 
dollars  to  be  spent  on  materiel  management  systems,  as  well  as  the 
appropriated  monies  already  invested,  bring  positive  results.  Specifically, 
Defense  did  not  first  conduct  economic  and  risk  assessments  that  would 
ensure  that  the  new  strategy  would  be  cost-effective  and  beneficial.  It  also 
did  not  incorporate  into  the  new  strategy  efforts  to  improve,  consolidate, 
and  privatize  logistics  operations.  Such  changes  will  impact  the  processes 
the  systems  are  being  developed  to  support.  Fiirther,  the  change  in 
strategy  was  not  justified  within  Defense’s  owm  oversight  process,  nor 
were  documents  critical  to  defining  the  program’s  objectives,  costs,  goals, 
and  risk  mitigation  strategies  prepared.  As  a  result.  Defense 
decisionmakers  were  not  afforded  an  opportunity  to  thoroughly  review  the 
new  program  before  deploying  new  systems. 

Moreover,  Defense  is  proceeding  with  deployments  rmder  the  new  strategy 
without  accommodating  the  time  required  for  testing  the  new  systems. 

This  greatly  increases  the  risk  that  Defense  will  experience  problems 
associated  with  shifting  testing  to  system  users  and  curtailing  the  levels  of 
testing  normally  done.  As  a  result.  Defense  is  likely  to  incur  substantial 
additional  costs  to  operate  and  maintain  legacy  systems  and  to  correct 
deficiencies  with  the  new  systems  that  sxuface  after  deployment  as  a 
result  of  delayed  testing. 
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Objectives,  Scope, 
and  Methodology 


unng  the  course  of  our  review  Defense  changed  its  strategy  for  materiel 
management  systems.  We  therefore  refocused  our  review  to  include 
evaluating  the  risks  associated  with  the  new  migration  strategy  and  the 
extent  to  which  it  will  facilitate  improvement  of  dod’s  materiel 
management  operations.  We  reviewed  dod’s  directives,  instructions  and 
gmdance  for  developing  and  implementing  systems  under  the  cim 
imtiatives,  as  these  projects  relate  to  the  materiel  management  business 
area.  We  mterviewed  officials  responsible  for  the  old  and  new  materiel 
man^ement  strategies  as  well  as  those  in  charge  of  developmental  testing 
^d  those  who  participated  in  the  early  deployment  of  some  applications. 
We  also  analyzed  system  design  documents,  program  assessments, 
acquisition  methodologies,  and  strategies.  Our  audit  was  performed  from 
January  through  May  1996  in  accordance  with  generally  accepted 
gwemment  auditing  standards.  We  performed  our  work  primarily  at  the 
offices  of  the  Deputy  Under  Secretary  of  Defense  for  Logistics  (dusd(L))  in 
Washington  D.C.;  the  Joint  Logistics  Systems  Center  (jlsc)  at 
Wri^t-Patterson  Air  Force  Base,  Ohio;  and  the  mmss  prototype  site  at  the 
Marine  Corps  Logistics  Base,  Albany,  Georgia.  Appendix  I  details  our 
scope  and  methodolo^.  The  Deputy  Under  Secretary  of  Defense  for 
Logistics  provided  written  comments  on  a  draft  of  this  report.  These 
cor^ents  are  discussed  at  the  end  of  this  report  and  presented,  along 
with  our  evaluation,  in  appendix  II. 


Background  Materiel  management  involves  determining  the  type  and  amount  of 

consumable  and  reparable  items  needed  for  daily  military  operations,  such 
^  animation,  fuel,  paint,  and  spare  parts;  purchasing  these  materials 
from  pnvate  vendors  or  manufacturing  agencies  within  dod;  and  tracking 
matenals  from  their  purchase  to  end  use.  Materiel  management  business 
operations  mcorporate  four  major  business  activities— asset  management 
reqrnrements  determination,  supply,  and  technical  data  support.  Annually' 
the  Department  spends  about  $19  billion  on  materiel  management 
operations,  and  it  cu^ently  manages  a  reported  6.3  million  inventory  items 
valued  at  a  reported  $73.6  bilhon. 


Materiel  Management 
Business  Strategies  Are 
Changing 


DOD  s  worldwide  logistics  operation  includes  about  1,400  warehouses  at  27 
drstnbution  depots  and  other  locations  that  provide  supplies  such  as 
electromcs,  construction,  and  industrial  items  to  the  military  services  The 
militaiy  services  use  large  amounts  of  these  items  to  maintain  and  repair 
weapon  systems  and  other  equipment.  For  example,  the  three  services 
operate  a  total  of  about  26  facilities  that  perform  detailed,  time-consuming 
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maintenance  of  major  weapon  system  components  such  as  radars, 
navigation  aids,  and  various  types  of  communication  equipment.  The 
services  also  store  supplies  (known  as  retail  inventory)  in  warehouses  at 
or  near  these  maintenance  facilities.  From  these  warehouses,  supplies  are 
distributed  to  the  mechanics  or  end-users  when  needed.  The  mechanics 
also  hold  some  of  these  same  items  in  nearby  storage  bins. 

We  have  previously  reported^  that  dod’s  large  inventory  levels  reflect  the 
management  practice  of  buying  and  storing  supplies  at  both  wholesale  and 
retail  locations  to  ensure  that  they  are  available  to  customer— sometimes 
years  in  advance  of  when  actually  needed,  dod  often  stores  inventories  in 
as  many  as  four  different  layers  between  suppliers  and  end-users.  Stormg 
inventory  at  many  different  locations  results  in  inventory  that  turns  over 
slowly,  which  produces  large  amounts  of  old,  obsolete,  and  excess  items. 
dod’s  multilayered  supply  system  also  increases  the  amount  of  supply  on 
hand  and  drives  up  the  cost  of  holding  inventory.  As  discussed  in  previous 
reports  in  1993  and  1994,®  this  is  a  philosophy  that  private  firms  have 
moved  away  from  in  an  attempt  to  lower  the  cost  of  doing  business, 
provide  better  service,  and  remain  competitive. 

For  example,  during  the  past  decade,  the  private  sector  instituted  a 
logistics  management  philosophy  that  provides  a  sharp  contoast  to  dod  s 
methods  of  managing  and  distributing  inventories.  Some  private  sector 
companies  have  avoided  inventory  management  problems  by  using 
modem,  “just-in-time”  business  practices  that  shift  responsibilities  for 
storing  and  managing  inventory  to  suppliers.  In  fact,  compames  that  are 
using  the  most  aggressive  practices  no  longer  store  inventory  in 
intermediate  locations  at  all;  their  suppliers  deliver  inventory  to  them  only 
when  needed.  Some  companies  have  achieved  large  savings  for  certain 
items  by  standardizing  items  being  used,  eUminating  bulk  storage  locations 
and,  most  importantly,  relying  on  prime  vendors  to  deliver  small  quantities 
when  and  where  needed. 

A  major  characteristic  of  these  new  logistics  practices  is  the  way  the 
companies  buy  supplies.  Companies  have  reduced  the  number  of  suppliers 


^DOD  Medical  Inventory:  Reductions  Can  Be  Made  Through  the  Use  of  Commercial  Practices 
fn  Dec.  5. 1991^  and  DOD  Food  Inventory:  Using  Private  Sector  Practices  Can  Reduce 

Costs  and  EUminate  Problems  (GAO/NSIAD-93-110,  June  4,  1993). 

3DOD  Food  Inventory:  Using  Private  Sector  Practices  Can  Reduce  Costs  and  Elinunate  Proble^ 
rnAn/MRTAn-O^-l  in.  June  4. 1993^:  Commercial  Practices:  Leading-Edge  Practices  Can  Help  DO _ 
Ltter  Manage  Clothing  and  Textile  Stocks  (GAO/NSIAD^94-64,  Apr.  13,  1994)^  Commerci^Pr^^ 
DOD  Could  Save  MUlions  by  Reducing  Maintenance  and  Repair  Inventon^  (GAO/NSlAD-yd-l&o, 

June  7, 1993). 
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they  use  by  establishing  long-term  agreements  with  only  a  few  key 
suppliers.  Typically,  suppliers  are  contracted  to  provide  a  company’s 
supplies  for  a  particular  commodity.  Thus,  most  of  the  management 
responsibilities  are  shifted  from  the  company  to  the  suppliers.  The 
suppliers  take  on  these  responsibilities  because  they  are  promised  a 
long-term  relationship  with  the  company.  Other  steps  companies  have 
taken  to  change  their  inventory  management  practices  include  using  direct 
dehveiy  programs,  primarily  through  the  use  of  a  prime  vendor .  By  using 
direct  delivery  programs,  companies  bypass  the  need  for  intermediate 
storage  and  handling  locations.  Once  the  end-users  order  supplies,  the 
suppliers  deliver  the  items  directly  to  the  user’s  facility  close  to  the  time 
when  the  items  are  needed.  Also,  to  facilitate  communication  channels 
with  the  suppliers,  electronic  ordering  systems  and  the  use  of  bar  coding 
are  often  used  to  eliminate  paperwork  and  speed  up  the  ordering  process. 

DOD  is  beginning  to  move  away  from  its  current  multilayered  inventory 
managernent  philosophy  by  developing  initiatives,  similar  to  those  adopted 
m  the  private  sector,  which  involve  some  combination  of  long-term 
contractmg  agreements,  direct  delivery  of  items  from  suppliers  to  the 
services,  and  electronic  data  interchange  for  streamlining  the  ordering 
process.  The  use  of  these  initiatives  is  allowing  dod  to  (1)  decrease 
procurement  lead  times,  (2)  increase  accuracy  in  forecasting  future  item 
demands,  (3)  reduce  paperwork,  and  (4)  reduce  inventory  levels.  While 
DOD  has  used  these  commercial  practices,  the  initiatives  generally  have 
been  hmited  in  scope  and  represent  only  a  small  portion  of  its  overall 
operations. 


Materiel  Management 
Systems  Intended  to 
Improve  Various  Business 
Processes 


Defense  wants  to  develop  and  deploy  materiel  management  systems  to 
improve  business  operations  and  processes  nationally  for  dod  materiel 
management  and  reduce  the  costs  associated  with  inventory,  personnel 
and  inefficient  systems.  Historically,  while  the  services  and  dla  have  had 
similar  logistics  activities,  they  employed  widely  different  processes  and 
supportmg  information  systems.  Currently,  Defense  relies  on  over  a 
reported  600  legacy  systems  to  carry  out  wholesale  logistics  operations  As 
these  systems  become  fragmented,  outdated,  and  inefficient,  they  require 
biUiom  of  dollars  in  maintenance  costs.  According  to  Defense,  because 
today  s  materiel  managers  do  not  have  access  to  timely,  accurate,  and 
rehable  logistics  information,  they  increasingly  make  unnecessary 
requisitions,  which,  in  turn,  result  in  excess  inventory  and  waste.  In 
addition,  according  to  Defense,  fragmented  systems  and  the  lack  of 
current  technology  have  severely  affected  the  ability  to  achieve  greater 
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asset  visibility;  quickly  adjust  requirements  for  materiel  supply  items  to 
support  military  operations;  standardize  planning,  requisitioning,  and 
inventory  control;  and  provide  greater  support  with  fewer  resources. 

By  embarking  in  1992  on  a  strategy  to  develop  the  materiel  management 
standard  system  (mmss).  Defense  sought  to  replace  hundreds  of 
service-unique  legacy  systems  being  used  to  acquire,  manage,  move,  and 
maintain  inventory  items  with  nine  standard  systems.^  These  are  the 
(1)  Central  Secondary  Item  Stratification  System,  (2)  Configuration 
Management  Information  System,  (3)  Deficiency  Reporting  System, 

(4)  Initial  Requirements  Determination/Readiness  Based  Sparing  System, 

(5)  Maintenance  Planning  and  Execution  System,  (6)  Production 
Definition  Support  System,  (7)  Provisioning  Cataloging  Technical  Support 
System,  (8)  Requirements  Computation  System,  and  (9)  Stock  Control 
System.  The  specific  functions  of  each  of  these  nine  systems  are  described 
in  appendix  HI.  Generally,  these  systems  are  intended  to  improve  business 
operations  in  the  following  ways: 

•  Asset  management — ^provide  greater  asset  visibility  from  the  time  of 
purchase  to  use  and  the  capabiUty  to  track  and  monitor  product  quality 
using  automated  deficiency  reports  during  the  wholesale  process. 

.  Requirements  determinations — ^better  define  initial  and  repair 

requirements  for  supply  items  based  on  readiness  scenarios  and  automate 
the  computation  of  repair  schedules  and  budgets. 

.  Supply  and  technical  data— automate  paper  copy  guidebooks,  procedures, 
and  regulations  needed  to  catalog  new  inventory  items  and  provide 
managers  with  greater  configuration  control  of  inventory  items. 


Defense  referred  to  this  program  as  the  “Big  Bang”  strategy  because  it 
involved  installing  the  entire  suite  of  applications  at  each  of  the  17 
inventory  control  points,  rather  than  deploying  each  application  as  it  was 
developed.  The  new  systems  were  to  be  integrated®  so  that  the  services 
and  DLA  could  communicate  and  exchange  data  with  each  other  and  across 
business  activities  outside  materiel  management,  such  as  finance, 
procurement,  personnel,  and  logistics,  jlsc  planned  to  field  mmss  across  ^1 
17  inventoiy  control  points  and  have  the  system  be  fully  implemented  m  7 


^JLSC,  in  cooperation  with  teams  of  service  and  DLA  experts,  imtiaUy  f  ^4  im^ati^^^^ 

(now  caUed  applications  by  JLSC)  to  form  MMSS.  As  a  result  of 

Ltween  existing  appUcations,  the  number  of  system  applications  has  been  reduced  from  24  to  10.  ta 
March  1995,  JLSC  terminated  the  development  of  another  appUcataon,  pnmanly  as  a  result  of  fundmg 
cuts,  bringing  the  total  to  the  current  nine  applications. 


5DOD  defines  integration  as  a  process  of  combining  software  and/or  hardware  components  into  an 
overall  system. 
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to  8  years,  jlsc  expected  this  standard  system  to  save  billions  of  dollars  in 
logistics  costs  by  consolidating  and  streamlining  management  operatioirs, 
improvmg  the  responsiveness,  accuracy,  and  timeliness  of  data,  and 
eliminating  the  cost  of  maintaining  some  information  systems  that  support 
the  same  business  processes. 


Problems  in  the 
Materiel  Management 
Migration  Strategy 


From  1992  to  late  1995,  Defense  spent  about  $714  million  developing 
standard  systems  with  minimal  results.  During  that  time,  there  were 
dramatic  changes  in  the  goals  and  expectations  for  the  program  and  only 
one  application  was  partially  deployed.  Because  of  changes  in  objectives 
and  schedulmg  and  problems  in  development,  prospects  for  achieving  the 
onginal  objective  of  implementing  a  standard  suite  of  integrated  materiel 
manageinent  systems  appeared  dim.  At  the  same  time,  the  services  and  dla 
were  asking  for  quicker  system  deployments. 


^  table  1  shows  Defense  began  the  migration  strategy  in  1992  with  the 
mtent  of  implementing  an  integrated  mmss  system  in  7  to  8  years.  But  only 
a  year  later,  it  decided  to  implement  the  system  in  3  years.  About  2  years 
after  that,  the  program  was  completely  rebaselined  because  of  funding 
cuts,  cost  overruns,  schedule  slippages,  and  poor  contractor  performance 
During  the  same  time  period,  the  objectives  of  the  program  changed:  at  the 
start  of  the  program,  business  process  improvements  were  to  be  identified 
while  systems  were  under  development;  in  1993,  improvements  were  to  be 
Identified  after  unplementation.  Taken  together,  these  changes  raised 
senous  concerns  about  mmss  among  the  services  and  dla. 
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Date 

Decisions 

October  1989 

DOD  establishes  the  Corporate  Information  Management 
initiative  to  improve  business  practices,  make  better  use  of 
information  technology  and  eliminate  duplicate  administrative 
systems,  such  as  payroll,  medical,  and  materiel  management. 

November  1991 

DOD  directs  that  the  Joint  Logistics  Systems  Center  (JLSC)  be 
established  to  facilitate  the  improvement  of  materiel 
management  processes  by  identifying  business  process 
improvements  and  managing  the  development  and  deployment 
of  a  standard  materiel  management  system  to  replace 
service-unique  systems  currently  used. 

November  1992 

DOD  directs  JLSC  to  focus  on  selecting  standard  logistics 
information  systems  from  existing  legacy  systems  called 
migration  systems.  JLSC  selects  24  migration  systems  to  form 
MMSS.  The  suite  of  systems  were  to  be  implemented  at  17 
control  points  in  7  to  8  years.  JLSC  estimated  savings  of 
approximately  $12  billion  over  a  10-year  period. 

October  1993 

DOD  changes  JLSC  priorities  by  directing  it  to  field  an 
integrated  standard  MMSS  migration  system  in  3  years. 

Business  process  improvements  and  development  of 
standardized  data  were  to  follow  implementation. 

January  1995 

JLSC  informs  DUSD(L)  that  the  3-year  requirement  is  an 
unrealistic  time  frame  for  developing  applications  and  that 
funding  cuts  (a  reported  $320  million  in  fiscal  years  1995 
through  1997)  made  it  difficult  to  meet  the  deadline. 

July  1995 

JLSC  rebaselines  entire  MMSS  program,  expecting  full 
implementation  in  2002. 

July  1995 

A  team  of  contractors  hired  to  compiete  an  economic  analysis 
to  determine  the  most  cost-effective  MMSS  approach 
recommends  that  all  nine  applications  be  fielded 
simultaneously  as  a  single  entity.  However,  because  of  funding 
constraints,  JLSC  chose  to  adopt  an  incremental  approach  and 
field  MMSS  first  as  an  interfaced  system  and  then  implement  it 
as  an  integrated  system.  The  contract  team  estimated  that  the 
system  would  cost  about  $4.8  billion  to  complete  and  take  6 
years  longer  to  fully  deploy  (fiscal  year  2005  compared  to  fiscal 
year  1999).  JLSC  estimated  savings  of  about  $15  billion  (a 
return  of  $4.5  for  every  dollar  spent)  over  a  15-year  period 
starting  in  fiscal  year  1998  from  (1)  reduced  inventory  costs,  of 
which  nearly  89  percent  would  come  from  improved  processes 
and  procurement  computations,  (2)  reduced  direct  and  indirect 
labor  costs  by  as  much  as  7  percent,  and  (3)  reduced  costs  of 
about  4  percent  as  a  result  of  shutting  down  inefficient  legacy 
systems. 

December  1995 

DUSD(L)  acknowledges  that  the  original  strategy  cannot  be 
completed  as  planned  without  major  additional  investments  in 
time  and  funding  to  correct  development,  schedule,  and 
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For  the  one  application  that  was  deployed  (the  Stock  Control  System),  the 
development  and  scheduling  problems  were  particularly  evident.  As 
appendix  V  further  details,  jlsc  shifted  system  testing  onto  the  useis  in 
order  to  meet  milestone  dates.  One  official  told  jlsc  that  its  scheduling  of 
testing  actually  extended  time  frames  and  resulted  in  a  loss  of  confidence 
from  users.  This,  coupled  with  other  scs  problems  related  to  resolving 
deficiencies  discovered  during  user  testing,  poor  training,  and  inadequate 
system  documentation,  prevented  the  application  from  providing  the 
benefits  originally  anticipated. 

Because  of  the  development  and  scheduling  problems,  the  services  and 
DLA  reported  serious  reservations  about  implementing  several  of  the  new 
systems  because  they  believed  that  some  of  their  existing  legacy  systems 
were  better  than  the  planned  standard  systems.  They  concluded  that  in 
some  cases,  the  new  systems,  such  as  the  Stock  Control  System  and  the 
Imtial  Requirements  Determination/Readiness  Based  Sparing  system, 
either  would  not  meet  their  operational  requirements  or  lacked  the 
necessary  functionality  to  allow  them  to  shut  down  existing  legacy 
systems  as  planned.  Nevertheless,  the  services  and  dla  claimed  that  some 
of  their  legacy  systems  were  quickly  deteriorating  and  that  they  could  not 
fimd  necessary  up^ades.  Therefore,  they  demanded  deployment  of  the 
new  systems  as  quickly  as  possible  based  on  their  individual  service  needs. 

April  1995,  a  Defense  evaluation  team,  comprised  of  representatives 
from  the  Office  of  the  Secretary  of  Defense,  jlsc,  dla,  the  military  services 
and  independent  contractors,  reviewed  selected  dod  inventory  control 
point  processes  and  concluded  that  the  migratory  systems  approach  to 
standardizing  and  upgrading  materiel  management  automated  data 
processing  systems  in  dod  is  not  workable.  The  team  recommended  that 
JLSC  discontinue  its  current  efi'orts  to  develop  mmss  and  advised  the 
Secretary  of  Defense  to  redirect  jlsc,  the  services,  and  dla  toward  a 
long-term  effort  to  develop  a  unified  automated  data  processing  supply 
system  using  an  independent  contractor  to  design,  develop,  and  prototype 


Nevertheless,  in  December  1996,  because  of  pressure  from  the  services 
and  DLA  and  the  problems  they  were  experiencing  with  the  mmss  migration 
strategy,  dod  dramatically  changed  the  mmss  scope  and  implementation 
approach.  Program  officials  believed  that  if  systems  were  not  deployed 
quickly,  the  entire  materiel  management  system  program  would  be 
vulnerable  to  additional  funding  cuts,  thus  jeopardizing  the  entire  program 
and  nsking  total  failm-e.  The  commander  of  jlsc  stated  that  if  customers 
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did  not  see  immediate  results,  his  organization  would  be  in  danger  of 
“going  out  of  business.”  In  order  to  accelerate  deployments  under  the 
revised  approach,  jlsc  no  longer  plans  to  deploy  a  standard  materiel 
management  system.  Instead,  it  will  deploy  applications  incrementally  as 
they  are  developed. 


New  Materiel 
Management  Strategy 
Faces  Serious  Flaws 


In  December  1995,  Defense  embarked  on  an  accelerated  deployment 
strategy  of  the  nine  applications  that  make  up  mmss  in  order  to  meet 
services  and  dla  priorities  and  to  realize  operational  benefits  sooner  than 
originally  planned.  Under  this  strategy,  jlsc  will  no  longer  deploy  an 
integrated  suite  of  standard  mmss  systems.®  Rather,  it  now  plans  to 
individually  deploy  each  of  the  nine  system  applications  as  they  are 
developed  at  selected  sites  from  fiscal  year  1996  through  fiscal  year  1999. 
The  services  and  dla  will  choose  which  applications  they  want,  when  and 
where  they  will  be  deployed  and,  as  a  result,  some  inventory  control 
points  may  never  receive  new  systems.  Deployment  will  be  co^trained  by 
available  funding,  jlsc  and  dusdcl)  refer  to  the  new  strategy  as  “deploy  or 
die”  sinc6  prograiti  officisls  b6li6V6d  that  unlGss  thGSG  systGins  WGrG 
deployed  quickly,  the  entire  materiel  management  system  program  would 
be  vulnerable  to  additional  funding  cuts,  thus  placing  the  program  in 
serious  jeopardy  of  total  failure.  The  current  deployment  schedule  of  these 
systems  is  provided  in  appendix  IV.  Table  2  reflects  the  differences  in  the 
two  strategies. 


•*In  March  1996,  DUSD(L)  eliminated  the  term  “Standard”  from  the  MMSS  name.  MMSS  is  now  called 
Materiel  Management  Systems  to  reflect  of  the  current  situation  and  new  program  strategy. 
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Table  2:  Comparison  of  Old  and  New 
Materiel  Management  Strategies 


Big  bang 
strategy  (Old) 

Deploy  or  die 
strategy  (new) 

Deployment 

As  an  integrated  suite  of  nine 
standard  applications 

Nine  applications  deployed 
separately  and  not  as  a 
standard  suite 

Suites  installed  at  all  inventory 
control  points,  one  at  a  time 

Applications  installed 
simultaneously  at  multiple  sites 
and  at  some  inventory  control 
points — based  on  need 

Suites  installed  at  additional  sites  Applications  installed  at  many 

service-  unique  and  unplanned 

—  sites 

Deployed  after  developmental 
and  Integration  testing  is 
complete 

Deployed  “as  is”  before 
developmental  and  integration 
testing  is  complete 

Satisfies  CIM  objectives  of 
standard  applications  and  data 

Does  not  satisfy  CIM  objectives 
since  applications  and  data  will 
not  be  standardized 

Cost  estimate 

$5.3  billion  life  cycle 

To  be  determined 

Schedule 

Complete  deployment  from 
fiscal  year  1997  through  fiscal 
year  2005 

Begin  accelerated  deployment 
of  individual  applications  in 
fiscal  year  1996 

To  be  determined;  deployments 
from  fiscal  year  1997  through 
fiscal  year  1999  depending  on 
fundina 

Benefits 

$15  billion  over  15-year  life  cycle 

Unknown  but  believed  to  be 
significantly  less  than  $15  billion 

Legacy  appiications  Hundreds  to  be  turned  off 

To  be  determined — but 
significantly  slower  and  fewer 

In  turning  to  the  new  materiel  management  systems  strategy,  Defense  is 
now  mtent  on  delivering  new  applications  to  customers  as  soon  as 
possible.  But  this  haste  puts  the  new  materiel  management  systems 
development  at  higher  risk  than  the  previous  one.  As  the  following 
sections  discuss,  Defense  will  begin  deploying  new  systems  before  it 
clearly  defines  its  approach,  ensures  adequate  oversight,  and  plans  for 
economic  and  techmcal  risks.  Defense  wiU  also  begin  deployments 
without  considering  the  effects  of  m^or  upcoming  changes  to  materiel 
management  operations.  In  addition.  Defense  will  be  deploying  all 
^plications  before  critical  necessary  testing  is  complete.  We  believe  that 
these  steps  ^e  all  critical  to  ensuring  that  Defense  gets  the  most  from 
each  dollar  it  invests  in  materiel  management  systems.  If  Defense  neglects 
to  address  them,  it  will  likely  incur  substantial  additional  costs  associated 
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Avith  maintaining  legacy  systems,  interfacing  them  with  the  new  systems, 
funding  the  rework  to  correct  problems  surfacing  after  deployment,  and 
adapting  its  approach  to  expected  dramatic  changes  in  operations  and 
systems. 

New  Strategy  Still 
Lacks  Direction  and 
Justification 

'  Under  the  new  strategy,  services  will  be  keeping  their  legacy  systems 
longer  than  anticipated  and  many  will  not  be  shut  down.  This  is  a  major 
departure  from  Defense’s  previous  goal  of  eliminating  hundreds  of 
redundant  legacy  systems  and  varied  business  processes  in  order  to  move 
to  standard  integrated  systems  and  processes.  Yet  for  such  a  significant 
change  in  direction.  Defense  first  did  not  conduct  assessments  that  would 
ensure  that  the  strategy  would  be  cost-effective  and  beneficial.  It  also  did 
not  incorporate  into  the  strategy  plans  to  consolidate  and  privatize 
operations  and  other  alternatives  being  considered  to  enhance  existing 
systems.  In  addition,  the  change  was  not  justified  within  the  Department’s 
own  oversight  process,  nor  were  documents  critical  to  defining  the 
programs  objectives,  costs,  goals,  and  risk  mitigation  strategies  prepared. 

Economic  and  Risk 
Assessments  Not 
Completed 

jLSC  is  proceeding  with  the  new  strategy  without  first  conducting  critical 
economic  and  risk  assessments  that  would  estimate  project  costs, 
benefits,  and  risks  and  evaluate  system  choices  based  on  these  analyses. 
Without  these  assessments,  dod  has  no  assurance  that  the  best  or  inost 
cost-effective  systems  are  selected  for  migration  nor  can  it  plan  actions 
designed  to  avoid  or  lessen  the  potential  for  project  delay ,  oyeispending, 
or  failure.  These  evaluations  are  particularly  important  at  this  time 
because,  according  to  program  officials,  the  estimated  mmss  lifecycle  costs 
and  expected  benefits  in  the  July  1996  economic  analysis  do  not  reflect  the 
most  recent  strategy  change.  These  evaluations  would  also  help  dod  m 
planning  to  mitigate  some  of  the  additional  costs  associated  with 
maintaining  legacy  systems  that  will  be  incurred  as  a  result  of  the  new 
strategy. 

The  benefits  of  dod  using  these  analyses,  for  example,  could  have  been 
realized  in  choosing  the  systems  to  deploy  first  in  the  new  strategy.  As 
table  3  shows,  three  of  the  four  systems  (the  Configuration  Management 
Information  System  (cmis),  the  Product  Definition  Support  System  (pdss), 
and  the  Deficiency  Reporting  System  (drs))  scheduled  for  deployment  m 
fiscal  year  1996  have  very  low  projected  benefits.  The  benefits  listed  m 
table  3  were  taken  from  the  July  1995  economic  analysis. 
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Table  3:  Benefits  Resulting  From 
Implementation  of  Systems 


System 


Requirements  Computation  System 
(RCS) 

Stock  Control  System  (SCS) 


Dollar  benefits 
(Then  year 
dollars  in 
_ millions) 


Percent  of 
total 


Priorities 
under  new 
strategy 


$5,171 


36.6 


Maintenance  Planning  and  Execution 
(MP«S(E) 


2,953 


20.9 


Initial  Requirements 
Determination/Readiness  Based 
Sparing  (IRD/RBS) 


2,027 


14.3 


Provisioning  Cataloging  Technical 
Support  System  (PCTSS) 

Configuration  Management 
Information  System  (CMIS) 


1,637 


11.6 


X 


1,096 


7.8 


Central  Secondary  Item  Stratification 
(CSIS) 

Product  Definition  Support  System 
(PDSS) 

Deficiency  Reporting  System  (DRS) 


555 


3.9 


277 


2.0 


240 


1.7 


Total 


185 


1.3 


X 


100 


Note:  The  benefits  noted  in  this  table  were  not  validated  by  DOD. 

approximately  $845  million  in  expected  benefits  from  software 
support  and  the  Simultaneous  Multi-indenture  Multi-echelon  Computation 
system  which  was  terminated  in  March  1995  due  to  funding  cuts.  ^ 


According  to  pro^am  officials,  ird/rbs,  cmis,  pdss,  and  des  were  chosen  for 
deployment  first  in  the  new  strategy  because  they  are  further  along  in 
development.  However,  we  believe  that  had  Defense  analyzed  costs, 
beriefits,  and  rfeks  associated  with  all  selections,  it  would  have  had  to 
seriously  consider  whether  the  benefits  associated  with  rcs,  scs  and  mp&e 
made  it  imperative  to  concentrate  on  their  development  first. 


Adjtionally,  by  not  analyzing  and  anticipating  costs  and  risks  associated 
with  the  new  strategy,  jlsc  officials  told  us  that  they  do  not  know  how 
much  It  will  cost  to  maintain  the  legacy  systems  that  wiU  remain  under  the 
new  strategy  and  what  it  wiU  cost  to  interface  the  new  applications  with 
the  legacy  systems.  Because  these  systems  will  not  be  deployed  as  an 
integrated  suite  at  all  inventory  control  points,  the  services  and  dla  will 
have  to  operate  many  of  their  legacy  systems  for  a  substantially  longer 
penod  of  time.  In  turn,  the  large  number  (and  complexity)  of  interface 
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designs  is  likely  to  increase  development  and  deployment  costs 
significantly  and  delay  implementation  schedules.  In  November  1995,  jlsc 
reported  that  the  identification  and  development  of  more  than  3,000 
interfaces  to  existing  legacy  systems  in  support  of  multiple  deployments  is 
the  prime  technical  risk  facing  the  program. 

Finally,  economic  and  risk  analyses  would  reveal  potential  conflicts 
between  available  funding  and  planned  scheduling  of  deployments. 
According  to  the  mmss  program  manager,  the  number  of  actual 
deployments  for  both  fiscal  years  1996  and  1997  will  be  contingent  on 
available  funding.  In  April  1996,  the  manager  reported  that  the  revised 
fiscal  year  1997  schedule  is  too  ambitious  given  the  funding  projected  to 
be  available. 


Impact  of  Potential 
Changes  to  Materiel 
Management  Program  and 
Systems  Not  Assessed 

•  Recent  dod  initiatives  focus  on  privatizing  materiel  management 
operations  or  consolidating  inventory  control  points.  For  example,  the 
Commission  on  Roles  and  Missions  of  the  Armed  Forces  recommended,  in 
May  1995,  that  Defense  outsource  materiel  management  activities  relating 
to  cataloging,  inventory  management,  and  warehousing.  H  outsourcing 
occurs.  Defense  may  end  up  spending  millions  of  dollars  on  systems  for 
functions  that  are  later  outsourced  or  on  inventory  control  points  that  are 
later  consolidated. 

.  As  discussed  in  the  background  section  of  this  report,  dod  is  beginnmg  to 
move  away  from  its  multilayered  inventoiy  management  philosophy  by 
embarking  on  initiatives  similar  to  those  adopted  in  the  private  sector, 
which  involve  some  combination  of  long-term  contract  agreements,  direct 
delivery  of  items  ft^om  suppliers  to  the  services,  and  electronic  data 
interchange  for  streamlining  the  ordering  process.  These  initiatives  have 
not  been  a  part  of  the  system  migration  strategy;  however,  as  they  are 
expanded,  they  will  significantly  impact  the  processes  the  systems 
support. 

.  According  to  program  officials.  Defense  is  considering  implementmg  a 
“data-focused  approach”  to  materiel  management  systems  starting  in  fiscal 
year  1998,  which  would  enhance  interoperability  and  logistics 
modernization  efforts  through  the  use  of  “middleware”  software. 


In  addition  to  not  assessing  economic  and  technical  risks.  Defense  has  not 
assessed  the  impact  that  a  number  of  potential  changes  under 
consideration  for  material  management  operations  and  systems  could 
have  on  the  program.  These  changes,  and  their  implications,  include  the 
following. 
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New  Strategy  Lacks 
Adequate  Oversight 


Midc^eware  pei^ts  an  application  to  see  the  data  stored  in  other 
applications  as  if  they  were  in  a  single,  logical  data  repository.  In  doing  so 
It  precludes  the  need  to  radically  redesign  the  legacy  systems  and 
implement  data  standardization.  If  pursued,  the  middleware  alternative 
could  extend  deployment  schedules  and  drive  up  maintenance  costs  for 

eastmg  systems.  It  also  will  not  result  in  the  consolidation  or  eUmination 
Of  legacy  systems. 

Through  its  own  oversight  process  for  major  information  system  projects 
DOD  has  established  a  basis  by  which  decisionmakere— who  make  up  the  ’ 

ajor  Automated  Information  System  Review  Council  (maisrc) _ can 

ensme  that  sound  business  practices  are  followed  for  m^or  information 
technology  system  mvestments.  Under  maisrc  guidelines,  a  project  should 
be  reviewed  and  approved  at  each  of  five  decision  milestones  before 
substantial  funds  are  obligated.®  An  important  aspect  of  the  review 
process  is  that  it  lays  the  groundwork  for  ensuring  that  m^'or  initiatives 
are  clearly  defined,  user  requirements  will  be  met,  and  sound  acquisition 
and  testmg  strategies  are  in  place.  Documents  that  the  Council 
reviews— such  as  the  mission  needs  statement  and  the  acquisition  and  test 
ptos-justify  the  program’s  existence  and  define  economic  and  technical 

X^ioivS« 

In  Jmua^  1996,  the  Deputy  Assistant  Secretary  of  Defense  for  Command 
on  o ,  Commimcations,  and  Intelligence  placed  the  mmss  project  under 
the  MAISRC  oversight  review  process  for  the  first  time.  However,  the 
decision  to  make  such  a  drastic  change  for  the  materiel  management 
strategy  m  the  first  place  was  never  presented  to  maisrc.  As  a  result  key 
efense  decisionmakers  did  not  have  a  chance  to  evaluate  the  program  in 
order  to  decide  whether  to  continue  the  current  program,  make  minor 
changes,  redirect,  or  terminate  the  program  before  it  began. 

men  JLSC  entered  the  review  process  in  April  1996  (before  a  working-level 
team  versus  the  high-level  Council),  it  requested  approval  to  continue 
incrementally  fielding  individual  applications,  but  did  not  have  strategic 
plans  and  other  reqmred  documentation  that  the  Council  could  use  to 
reach  a  decision.  Based  on  the  working-level  team’s  review,  the  Council 
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withheld  authorization  to  proceed  until  certain  documents  were  submitted 
and  approved. 

In  its  May  13, 1996,  decision  memorandum,  the  Council  directed  the  mmss 
program  manager  to  prepare  the  basic  documents  required  for  maisrc 
review  over  the  next  180  days.  These  include 

.  a  mission  need  statement,  which  sets  the  goals  of  the  program  and  defines 
projected  capabilities  and  needs  in  broad  operational  terms; 

.  an  acquisition  strategy  to  guide  the  entire  acquisition  process  throughout 
the  system  development  life  cycle  and  serve  as  the  framework  for 
planning,  directing,  and  managing  the  program, 

.  an  operational  requirements  document  to  document  user  objectives  and 
minimum  acceptable  requirements  for  systems  and  to  become  the  basis 
for  operational  performance  criteria  and  testing;  and 
.  a  plan  for  preparing  an  economic  analysis. 

Primarily  to  enable  dod  to  meet  contractual  obligations,  the  Council  plans 
to  hold  another  working-level  session  to  evaluate  and  approve  the 
acquisition  and  test  and  evaluation  strategies  before  the  end  of  fiscal  ye^ 
1996.  According  to  the  Council,  if  these  strategies  are  approved,  jlsc  wiU 
be  authorized  to  proceed  with  deployment.  However,  when  this  decision  is 
made,  the  other  critical  documents,  such  as  the  plan  for  preparing  an 
economic  analysis  and  the  operational  requirements  document,  will  not  be 
available  for  the  Council’s  review.  Therefore,  jlsc  may  be  authonzed  to 
proceed  with  deployment  before  key  decisionmakers  at  Defense  have 
reviewed  a  cost-benefit  analysis,  funding  profile,  and  other  important 
information  that  would  shed  light  on  the  risks  and  costs  associated  wit 
the  new  strategy.  We  believe  that  the  risks  associated  with  the  new 
strategy  and  problems  experienced  with  the  old  strategy  warrant  a  full, 
high-level  Council  review  of  all  MAiSRC-related  documents  before 
deployments  proceed. 


Premature 
Deployment  Poses 
Additional  Risks  for 
the  New  Strategy 


The  new  deployment  schedule  for  materiel  management  systems  does  not 
accommodate  the  time  required  for  testing  the  new  systems.  In  fact,  all 
systems  scheduled  to  be  deployed  in  fiscal  year  1996  and  1997  wiU  only 
have  met  a  minimal  testing  level,  that  is,  developmental  testing  by  the 
contractor.  As  a  result,  the  risk  is  greatly  increased  that  Defense  will 
experience  problems  associated  with  shifting  testing  to  system  users  and 
curtailing  the  levels  of  testing  normally  done.  This  has  already  been  the 
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case  with  the  one  application  that  was  deployed  at  the  mmss  prototype 
site— the  Stock  Control  System. 

The  four  applications  scheduled  for  deployment  in  fiscal  year  1996  are  still 
m  the  latter  stages  of  development  and  have  yet  to  complete  required 
developmental  and  integration  testing.  In  an  attempt  to  meet  the  revised 
deployment  schedule,  jlsc  is  shifting  developmental  testing 
responsibilities  from  the  development  contractors  to  system  users  where 
t  e  application  is  to  be  initially  deployed.  In  some  instances,  jlsc  will  also 
forgo  system  qualification  and  integration  testing  altogether — which  are 
cntical  to  determining  whether  the  applications  will  work  as  planned. 
According  to  program  officials,  the  intent  is  to  demonstrate 
interoperability  of  the  database  concept  in  a  customer  environment  and  to 
obtain  customer  feedback  more  quickly. 

However,  not  successfully  completing  these  tests  prior  to  deployment 
increases  the  risk  that  software  problems  will  go  undetected  until  the  later 
phases  of  the  system  fifecycle.  According  to  the  Test  Director,  it  is  much 
more  expensive  and  time  consuming  to  correct  errors  once  the 
applications  are  operational.  Program  officials  also  acknowledge  that 
problems  detected  at  the  user  sites  will  be  more  expensive  to  fix  and  could 
offset  or  exceed  up-front  investment  savings. 

In  the  absence  of  an  approved  test  and  evaluation  masterplan,  jlsc  is 
negotiating  memorandums  of  agreement  (moa)  with  respective  users  to 
efine  test  conditions,  assumptions,  and  responsibilities.  Our  review  of  the 
one  approved  MOA  covering  the  Navy’s  deployment  of  the  cmis  application 
in  fiscal  year  1996  raises  concerns  about  the  test  program.  For  example 
the  MOA  shows  that  the  application  will  be  an  “as  is”  veision  which  has  not 
been  accepted  by  and  will  not  be  interfaced  with  any  legacy  systems 
dunng  the  test  period.  Although  the  Navy  is  required  to  prepare  a 
lesson-learned  report  after  it  completes  testing,  no  formal  test  plans  or  test 
reports  are  required  under  the  testing  process.  Without  these  key 
documents,  jlsc  has  little  assurance  that  all  necessary  tests  will  be 

completed  and  that  problems  encountered  with  the  system  are  thoroughly 
documented.  *  ^ 
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Stock  Control  System 
Illustrates  Problems 
Associated  With  Early 
Deployment  and  User 
Testing 


The  early  deployment  of  the  Stock  Control  System  application  under  the 
previous  migration  strategy  illustrates  problems  associated  with  shiftmg 
testing  to  users.  As  discussed  in  appendix  V,  the  first  mmss  prototype  site, 
the  Marine  Corps  Logistics  Base,  Albany,  Georgia,  was  activated  on  May  1, 
1996.  At  that  time,  jlsc  deployed  an  early  version  of  the  scs  application, 
which  had  only  about  50-percent  functionality  of  the  asset  management 
software.  The  Marine  Corps  expected  that  this  release  would  resolve  its 
core  asset  management  system  deficiencies  and  demonstrate  operational 
functionality  and  practical  business  process  improvements. 


As  of  May  1996,  according  to  system  users,  the  system  has  failed  to 
provide  substantial  improvement  over  the  legacy  applications  being  used 
at  Albany.  Because  the  project  contractor  delivered  the  application 
basically  untested,  with  very  limited  functionality  and  inadequate  user 
documentation,  the  Marine  Corps  has  had  to  perform  extensive  and  costly 
amounts  of  rework,  debugging,  and  on-site  testing.  The  Marine  Corps  has 
initiated  65  changes  to  correct  major  functional  deficiencies;  however, 
only  42  have  been  funded  to  date.  Until  the  remaining  deficierrcies  are 
corrected,  scs  will  not  be  able  to  meet  all  of  the  Marine  Corps 
reqrrirements. 

Because  of  continuous  problems  in  defining  requirements  and  schedule 
slippages,  jlsc  stopped  all  development  work  on  scs  in  December  1995.  At 
the  time,  scs  development  was  about  55  percent  complete.  In  May  1996, 
the  Logistics  Management  Institute  (a  contractor  hired  by  jlsc  to  provide 
technical  support)  recommended  that  jlsc  terminate  scs  development  and 
maintain  legacy  asset  management  systems  rather  than  invest  an 
additional  2  years  and  as  much  as  $100  million  to  correct  the  problems. 
JLSC  still  plans  to  deploy  scs;  however,  it  will  limit  additional  functional 
enhancements  and  will  deploy  the  system  only  to  the  Marine  Corps  and 
the  Air  Force. 


^  To  provide  service  on  demand,  Defense  made  a  mqjor  change  in  its 

ConclUSiORS  materiel  management  migration  system  policy.  In  doirig  so,  it  is  clearly  on 

a  course  to  accelerate  system  deployments  before  critical  steps  are  taken 
that  would  help  ensure  that  good  business  decisions  are  made  and  that 
risks  are  minimized.  As  a  result.  Defense  may  likely  deploy  systems  that 
will  not  be  significantly  better  than  the  hundreds  of  legacy  systems  already 
in  place,  and  it  could  waste  millions  of  dollars  resolving  problems  that 
result  from  the  lack  of  developing  and  implementing  a  clear  and  cohesive 
strategy.  Before  proceeding  with  any  new  strategy,  it  is  imperative  that 
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Defense  take  the  necessaiy  steps  to  fully  define  its  approach,  plan  for 
nsks,  ensure  adequate  oversight  and  complete  testing  of  the  new  systems. 


We  recommend  that  the  Secretary  of  Defense  stop  the  materiel 
management  system  development  and  deployment  until  (1)  dusd(L) 
completes  an  economic  analysis  and  a  comprehensive  implementation 
plan,  including  actions  to  be  taken,  schedules,  milestones,  and 
performance  measures,  and  a  technical  risk  plan  and  (2)  the  full  maisrc 
reviews  and  approves  these  plans. 


The  Department  of  Defense  provided  written  comments  on  a  draft  of  this 
report.  The  Deputy  Under  Secretary  of  Defense  for  Logistics  generally 
agreed  with  our  findings  but  disagreed  with  our  recommendations. 
Defense’s  specific  comments  are  summarized  below  and  presented  along 
with  our  rebuttals,  in  appendix  H. 

In  commenting  on  a  draft  of  this  report.  Defense  effectively  acknowledged 
that  the  first  materiel  management  strategy  failed  and  it  agreed  on  the 
need  to  mitigate  risks  confronting  the  new  strategy.  However,  it  did  not 
agree  with  our  recommendations  to  stop  materiel  management  system 
development  and  deployment  imtil  it  takes  necessary  steps  to  define  its 
approach,  plan  for  risks,  and  ensure  adequate  oversight.  Instead,  Defense 
believes  it  is  addressing  the  concerns  e3q)ressed  in  our  report  under  the 
logistics  business  systems  strategy  it  is  currently  developing.  The  latest 
strategy  focuses  on  creating  a  common  operating  environment  for 
logistics.  According  to  Defense,  under  the  common  operating 
enwonment,  guidelines  and  standards  specifying  how  to  reuse  existing 
software  and  build  new  software  will  facilitate  system  interoperability  and 
allow  for  continually  evolving  computer  capabilities. 

We  agree  with  the  Department’s  contention  that  its  materiel  management 
strategy  has  failed  and  commend  it  for  pursuing  alternative  strategies, 
such  as  privatization  and  developing  a  common  operating  environment 
based  on  commercial  off-the-shelf  (cots)  systems.  Such  alternatives  may 
well  solve  some  of  the  past  problems  associated  with  materiel 
management  systems.  However,  we  disagree  that  the  common  operating 
environment  strategy  being  developed  will  address  our  recommendations 
Without  first  conducting  required  economic  analyses.  Defense  has  no 
assurance  that  the  systems  it  is  currently  deploying,  which  were  selected 
under  the  failed  strategy,  will  fully  support  the  new  strategy  or  by 


Agency  Comments 
and  Our  Evaluation 


Recommendations 
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themselves  still  be  good  investments.  In  addition,  by  not  conducting  these 
analyses,  decisionmakers  will  lack  necessary  information  to  make  sound, 
informed  decisions  for  selecting  the  best  among  competing  alternatives 
and  xmderstanding  how  upcoming  major  changes  to  materiel  operations 
will  impact  their  strategy  and  alternatives.  These  shortcomings  led  to  the 
failure  of  the  first  strategy,  and  we  believe  that,  iinless  they  are  addressed. 
Defense  risks  failing  a  second  time. 

Further,  conducting  economic  and  risk  analyses  and  providing  for 
adequate  oversight  over  system  development,  is  not  only  required  by 
Defense’s  own  regulations  but  also  by  the  recently  enacted  Information 
Technology  Management  Reform  Act  (itmra),  which  took  effect  August  8, 
1996.  The  intent  of  this  legislation  is  to  prevent  failures  similar  to  &e 
material  management  standard  system  strategy.  Under  itmra,  dod  is 
required  to  design  and  implement  a  process  for  selecting  information 

technology  investments  using  criteria  such  as  risk-adjusted 

retum-on-investment  and  specific  criteria  for  comparing  and  prioritizing 
alternative  information  system  projects.  If  implemented  properly  as  part  of 
the  new  strategy.  Defense  can  have  a  means  for  senior  management  to 
obtain  timely  information  regarding  progress  in  terms  of  costs,  capabiUty 
of  the  system  to  meet  performance  requirements,  timeliness,  and  quality. 
Without  implementing  an  effective  investment  process  for  the  new 
strategy.  Defense  will  continue  to  risk  encountering  unmanaged 
development  risks,  low-value  or  redundant  information  technology 
projects,  and  too  much  emphasis  on  maintaining  old  systems  at  the 
expense  of  using  technology  to  redesign  outmoded  work  processes. 


We  are  sending  copies  of  this  report  to  the  Senate  Committee  on 
Governmental  Affairs  and  the  House  Committee  on  Government  Reform 
and  Oversight;  the  Secretaries  of  Defense,  the  Army,  the  Navy,  the  Air 
Force;  the  Director  of  the  Defense  Logistics  Agency;  the  Director  of  the 
Office  of  Management  and  Budget;  and  other  interested  parties.  Copies 
will  be  made  available  to  others  on  request. 
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If  you  have  any  questions  about  this  report,  please  call  me  at 

(202)  612-6240  or  Carl  M.  Urie,  Assistant  Director,  at  (202)  512-6231.  Major 

contributors  to  this  report  are  listed  in  appendix  VI. 


Jack  L.  Brock,  Jr. 

Director,  Defense  Information 
and  Financial  Management  Systems 
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Objectives,  Scope,  jind  Methodology 


As  mandated  by  the  National  Defense  Authorization  Act  for  Fiscal  Year 
1996  (P*ublic  Law  104-106),  we  reviewed  the  Department  of  Defense’s 
Materiel  Management  Standard  System  (mmss).  The  original  objectives  of 
our  review  were  to  determine  (1)  the  mission  and  the  economic  and 
technical  basis  for  selecting  mmss  as  the  migrating  system  and  (2)  the 
extent  to  which  this  strategy  has  or  -will  improve  dod’s  materiel 
management  operations.  To  accomplish  our  objectives,  we  (1)  interviewed 
prografn  officials  and  contractors  responsible  for  developing, 
implementing,  and  managing  mmss  projects,  (2)  reviewed  pertinent 
program  and  contractor  documentation  such  as  cost  performance  reports, 
metrics,  quarterly  reports  on  major  automated  information  systems, 
economic  analyses,  implementation  and  migration  plans,  and  the  test  and 
evaluation  plan,  (3)  examined  system  design  documents,  program 
assessments,  and  acquisition  methodologies  and  strategies  to  support  the 
MMSS,  and  (4)  interviewed  senior  Defense  officials  responsible  for 
approving  and  directing  the  mmss  development  and  acquisition  regarding 
their  efforts  to  minimize  mmss  development  risks  and  improve  materiel 
management  operations. 

However,  shortly  after  we  began  the  review.  Defense  stopped  its  strategy 
to  develop  a  standard  materiel  management  system  and  started  making 
plans  to  separately  deploy  individual  applications.  Consequently,  we 
refocused  our  review  to  include  evaluating  the  risks  associated  with  this 
new  migration  strategy  and  the  extent  to  which  it  will  facilitate 
improvements  in  materiel  management  operations.  We  interviewed  dod 
and  program  officials  to  determine  the  rationale  behind  the  strategy 
change  and  the  potential  economic  and  technical  risks  threatening  the 
successful  implementation  of  the  new  strategy.  We  obtained  and  examined 
budgetary  and  cost  data,  reviewed  project  status  reports,  and  pertinent 
program  documents  such  as  the  revised  deployment  schedule,  test 
procedures,  and  program  decision  papers.  For  applications  scheduled  to 
be  deployed  in  fiscal  year  1996,  we  compared  the  test  procedures  with 
deployment  schedules  to  evaluate  potential  program  risks  of  deploying 
software  applications  before  successfully  completing  required  testing.  We 
were  hampered  in  our  attempt  to  assess  the  potential  improvements  to 
dod’s  materiel  management  operations  because  critical  strategic 
documents  such  as  the  revised  economic  analysis,  acquisition  strategy, 
and  mission  need  statement  had  not  been  completed.  To  determine  if  field 
locations  had  experienced  problems  resulting  from  insufficient  testing,  we 
also  interviewed  officials  at  the  Marine  Corps  Logistics  Base,  Albany, 
Georgia  who  had  participated  in  the  early  deployment  of  some  mmss 
applications. 
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Our  work  was  performed  from  January  1996  through  May  1996  in 
accordance  with  generally  accepted  government  auditing  standards.  We 
performed  our  work  primarily  at  the  offices  of  the  Deputy  Under  Secretary 
of  Defense  for  Logistics,  Washington  D.C.;  the  Joint  Logistics  Systems 
Center,  Wright-Patterson  Air  Force  Base,  Ohio;  and  the  Marine  Corps 
Logistics  Base,  Albany,  Georgia. 
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Comments  From  the  Department  of  Defense 


Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


ACQUISITION  AND 
TECHNOLOGY 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON  DC  20301-3000 


0  2  f  'J?  1398 


Mr.  Gene  L.  Dodaro 
Assistant  Comptroller  General 
Accounting  and  Information 
Management  Division 
U.S.  General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Dodaro: 

This  is  the  Department  of  Defense  response  to  the  General  Accounting  Office  (GAO) 
Draft  Report,  "DEFENSE  IRM:  Critical  Risks  Facing  New  Materiel  Management  Strategy," 
Dated  July  2, 1 996  (GAO  Code  5 1 1 344),  OSD  Case  1183.  The  Department  concurs  with  many 
of  the  concerns  expressed  in  the  report.  Concerted  action  has  already  been  initiated  by  the 
Department  to  address  these  concerns.  Because  these  actions  have  been  initiated,  the  Department 
does  not  concur  with  the  basic  GAO  recommendation  to  stop  all  materiel  management 
development  and  deployment  actions  and  develop  extensive  documentation  for  Major  Automated 
Information  Systems  Review  Council  (MAI SRC)  review. 

The  Department  is  addressing  the  concerns  expressed  by  the  draft  GAO  report  in  a  new 
logistics  business  systems  strategy  which  is  currently  being  written.  The  focus  of  the  new 
strategy  is  the  creation  of  a  common  operating  environment  for  logistics  within  the  Defense 
Information  Systems  Agency  (DISA)  Global  Combat  Support  System  (GCSS)  structure.  The 
magnitude  and  complexity  of  the  legacy  logistics  information  processing  environment  (including 
the  materiel  management  functions)  makes  the  task  of  executing  business  process  improvement 
at  a  macro  level  extremely  difficult.  Less  than  macro  level  improvements  are  equally  difficult  to 
execute  because  of  the  elaborate  web  of  interdependencies  in  coding  and  data  in  the  legacy 
systems.  To  create  the  conditions  in  which  manageable  business  improvement  can  occur,  a 
contemporary  technical  infrastructure  controlled  through  appropriate  configuration  management 
rules  is  needed.  The  Department  believes  that  the  GCSS  offers  that  Infrastructure. 

It  should  further  be  noted  that  ongoing  deployment  actions  have  been  approved  through 
the  MAISRC  structure  as  explained  in  the  attachment  hereto. 


Deputy  Under  Secretary 
of  Defense  (Logistics) 


Attachment 

o 
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Now  on  p.  19. 


See  comment  1 . 


Now  on  p.  2. 


GAO  DRAFT-DATED  JULY  2, 1996 
(GAO  CODE  511344)  OSD  CASE  1183 

DEFENSE  IRM:  Critical  Risks  Facing  New  Materiel  Management  Strategy 

DEPARTMENT  OF  DEFENSE  COMMENTS 

*4i*ic-k** 

RECOMMENDATION 


GAO  RECOMMENDATION:  recommend  that  the  Secretary  of  Defense  stop  materiel 

management  system  development  and  deployment  until  (1)  DUSD(L)  completes  an  economic 
analysis,  comprehensive  implementation  plan  to  include  actions  to  be  taken,  schedules, 
milestones,  and  performance  measures,  and  a  technical  risk  plan  and  (2)  the  full  MAISRC 
reviews  and  approves  these  plans,  **  (p.27/GAO  draft  report) 

The  basic  rationale  for  the  recommendation  was  summarized  by  the  GAO  as:  *\„Defense  did 
not  clearly  define  and  justify  its  approach;  plan  for  economic  and  technical  risks;  provide 
adequate  oversight;  or  consider  upcoming  major  changes  to  materiel  management  operations. 
In  addition,  Defense  is  proceeding  with  deploying  new  systems  before  developmental  testing  of 
new  systems  is  complete.  As  a  result,  Defense  is  likely  to  incur  substantial  additional  costs  to 
operate  and  maintain  legacy  systems  and  to  correct  deficiencies  with  the  new  systems  that 
surface  after  deployment  as  a  result  of  delayed  testing.  Further,  Defense  is  not  affording  its 
own  decisionmakers  an  opportunity  to  thoroughly  review  the  new  program  before  proceeding 
with  deployments.*^ {p.  4/GAO  draft  report) 

POD  RESPONSE:  Non  concur  with  recommendation;  concur  with  observations  about  past 
difficulties  and  the  need  to  mitigate  risks  which  potentially  exist  with  the  new  strategy. 

^..Jlefense  did  not  clearly  define  and  justify  its  approach,..**  GAO’s  perception  of  potential 
problems  with  the  revised  strategy  expresses  the  same  concerns  being  addressed  by  the 
Department  in  a  number  of  actions  which  are  currently  underway.  The  Services  and  DLA, 
facilitated  by  the  Logistics  Management  Institute,  recently  completed  an  assessment  of  the 
Material  Management  Standard  System  (MMSS).  On  May  23,  1996,  the  DoD  Logistics 
Information  Board  reviewed  the  results  of  the  assessment  and  recommendations.  Five 
alternative  strategies  were  offered  and  assessed  by  Service  and  DLA  representatives: 

Continue  the  “Big  Bang”  MMSS 

•  Incrementally  implement  as  designed 

•  Incrementally  implement  and  reengineer  the  “Core”  of  MMSS 
Shift  to  a  Common  Operating  Environment  and  Data  Driven  strategy 
Services  and  DLA  develop  and  maintain  independent  systems 
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See  comment  2. 


See  comment  3. 


There  is  strong  consensus  across  the  Services  and  DLA  to  pursue  a  strategy  which: 

•  Incrementally  evolves  from  Service  and  DLA  systems  to  a  modem  common  operating 
environment 

“  Accelerates  implementation  of  selected  materiel  management  applications  to  meet  the 
current  needs  of  the  Services  and  DLA,  and 

•  Facilitates  accelerated  modernization  of  business  processes  and  the  related  information 
systems  through  the  creation  of  a  common  operating  environment  and  standard  data. 

Logistics  Information  Board  members  have  agreed  that  the  standard  system  approach  is  no 
longer  appropriate  and  should  be  replaced  with  an  approach  which  is  founded  on  the 
implementation  of  a  common  operating  environment  and  data  driven  strategy.  This  refocus  was 
concurred  with  by  the  senior  logisticians  of  each  Service  and  DLA,  the  Deputy  Under  Secretary 
of  Defense  (Logistics),  the  Air  Force  Acquisition  Executive  (as  the  materiel  management 
acquisition  executive  agent),  and  the  OSD/C3L  Additionally,  the  Services'  perceived  need  to 
refocus  the  program  was  supported  by  an  independent  study  by  the  Institute  for  Defense 
Analysis. 

An  overall  definition  of  a  revised  logistics  business  systems  strategy  (including  materiel 
management)  is  being  documented  at  the  same  time  that  the  first  aspects  of  the  strategy  are  being 
executed.  It  is  anticipated  that  within  the  next  30  to  60  days  the  documented  strategy  will  have 
been  completed  and  published  and  that  work  will  have  begun  to  specify  the  plan  of  action  and 
milestones  which  will  derive  from  that  strategy.  Meanwhile,  action  to  deploy  both  technical 
infrastructure  and  new  functional  applications  is  proceeding  in  consonance  with  what  the  strategy 
will  state. 

The  core  of  the  strategy  will  be  the  establishment  of  a  common  technical/fiinctional  architecture 
within  which  logistics  business  applications  will  operate.  This  architecture  will  be  founded  on 
Defense  Information  Systems  Agency  (DISA)-prescribed  guidelines  and  tools  and  upon  a  jointly 
developed  data  strategy  which  is  currently  being  defined  as  a  subelement  of  the  LBS  strategy. 
Within  this  architecture  and  data  foundation,  the  definition  of  necessary  functional  applications 
resides  with  the  customers,  the  Military  Services  and  Defense  Agencies,  bounded  only  by  the 
need  to  comply  with  existing  and  emerging  OSD  policy  and  Common  Operating 
Environment/Defense  Information  Infrastructure  rules.  The  materiel  management  deployments 
occurring  in  FY  1996  were  requested  by  the  Military  Services  and  Defense  Agencies  as  a  result 
of  their  collective  review  of  the  applications  available  from  the  materiel  management  program 
manager. 

*\„planfor  economic  and  technical  risks...  ”  The  Economic  Analysis  recently  completed  in  July 
1 995  for  the  Materiel  Management  Standard  System  (MMSS)  addressed  program  risk. 

Although  this  document  does  not  reflect  the  current  strategy,  it  was  instrumental  in  highlighting 
issues  that  arc  being  addressed  in  the  new  strategy.  In  addition,  an  Economic  Analysis 
Working-level  integrated  Process  Team  (EA-WIPT)  has  been  established  to  address  the 
Economic  Analysis  Development  Plan  (EADP)  for  the  new  estimates  required  to  support  the  new 
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strategy.  This  approach  has  been  briefed  to  the  MAISRC  group  and  the  DOD  Components,  and 
all  concurred  with  this  approach.  The  first  meeting  of  the  EA-WIPT  was  held  on  25  June  1996, 
and  will  address  the  draft  plans  for  the  required  estimates  at  the  next  meeting. 

*\„provide  adequate  oversight,.  **  In  addition  to  the  acquisition  oversight  process  noted  above,  a 
related  Department  initiative  intended  to  assure  improved  corporate  management  of  the 
execution  of  the  new  strategy  and  customer  participation,  an  OSD-chaired  Logistics  Information 
Board  (LIB)  has  been  established  with  senior  membership  from  all  concerned  organizations  - 
Military  Services/Defense  Agencies,  OSD  policy  offices  and  JCS.  The  LIB  is  charged  with: 

1 .  Developing  a  Logistics  Business  Systems  Strategy, 

2.  Managing  the  actions  necessary  to  execute  the  strategy  in  terms  of  both  the  functional  and 
technical  goals  implicit  in  the  strategy,  and 

3 .  Assuring  the  appropriate  application  of  resources  to  the  accomplishment  of  those  goals. 

It  should  also  be  noted  that  the  MAISRC  staff  was  briefed,  several  working  level  integrated 
product  team  meetings  were  held,  and  the  MAISRC  principals  coordinated  on  the  May  13, 1996 
Acquisition  Decision  Memorandum  regarding  the  Materiel  Management  system. 

^'...consider  upcoming  major  changes  to  materiel  management  operations...  ”  It  is  the 
Department’s  contention  that  concerns  expressed  by  the  GAO  regarding  the  impact  of  other 
Departmental  initiatives,  such  as  privatization,  on  logistics  business  systems 
development/deployment  can  only  be  completely  evaluated  when  the  common  frame  of  reference 
to  be  offered  by  the  revised  architecture  and  data  strategy  is  complete.  In  the  interim,  the  JLSC 
is  pursuing  a  technical  strategy  that  will  allow  for  flexibility  and  interoperability  between  legacy 
and  COTS  environments.  The  tech  strategy  the  JLSC  MM  is  pursuing  adheres  to  the  DII/COE 
principles  and  addresses  the  reality  of  working  with  leading  COTS  systems  (the  potential  system 
of  choice  for  any  privatization  effort)  and  available  GOTS  solutions.  MM  systems  are  designed 
around  the  use  of  ORACLE  RDBMS  which  would  allow  a  COTS-based  system  to  reuse  the  MM 
data.  The  JLSC  believes  that  this  use  of  recognized  COTS/GOTS  packages  will  allow  it  the 
flexibility  to  adapt  to  evolving  environments  as  they  become  known.  In  addition,  all  known 
information  about  fimctional  outsourcing,  base  closure,  and  other  such  actions  is  considered 
before  deployment  decisions  are  made. 

**...is  proceeding  with  deploying  new  systems  before  developmental  testing  of  new  systems  is 
complete...fandJ...is  likely  to  incur  substantial  additional  costs.„to  correct  deficiencies  with  the 
new  systems  that  surface  after  deployment  as  a  result  of  delayed  testing...  ”  While  it  is  accurate 
to  state  that  deployments  are  occurring  before  "full  testing  is  completed",  it  is  important  to  note 
that  JLSC’s  strategy,  which  has  been  developed  with  the  DoD  test  community,  is  a  measured 
response  to  minimize  risk.  Specifically,  we  intend  to  field  “Alpha”  deployments  in  the  customer 
environments,  correct  deficiencies,  build  and  test  interfaces  while  moving  toward  a  Beta  version, 
leading  to  a  Site  Activation  Test  before  commencing  Operational  Testing.  This  approach  is  a 
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common  practice  in  the  commercial  world  to  mitigate  risk  and  reduce  cost  prior  to  full 
production. 

The  JLSC  is  working  closely  with  the  OSD  and  Army  testing  communities  and  is  close  to 
obtaining  MAISRC  approval  of  the  above  test  and  evaluation  strategy  for  materiel  management 
applications.  Further,  the  Department  does  not  agree  that  deployment  of  the  applications 
scheduled  in  FY  1996  and  to  be  scheduled  in  FY  1997  are  likely  to  "incur  substantial  additional 
costs...to  correct  deficiencies."  The  applications  being  deployed  are  adding  technology-based 
capabilities  to  business  processes  where  such  capabilities  do  not  exist  in  the  legacy  environment. 
While  the  technical  integration  of  these  new  capabilities  into  the  overall  Service/ Agency 
information  processes  will  add  to  their  respective  utilities,  they  will  bring  new  capabilities  in 
their  current  forms.  The  relative  value  of  these  new  capabilities  must  be  considered  in  judging 
when  deployment  should  occur.  All  application  processes  require  some  level  of  post-deployment 
maintenance  despite  the  degree  to  which  they  have  been  tested  prior  to  deployment.  While 
additional  testing  might  lower  the  amount  of  such  maintenance,  it  would  also  delay  the 
availability  of  the  new  capabilities  to  the  Services/Agencies.  It  is  the  Department’s  collective 
(i.e.,  OSD,  Services,  and  Agencies')  judgment  that  deployment  to  the  extent  allowed  by  available 
resources  and  consistent  with  customer  needs  should  proceed.  The  criteria  for  fielding 
information  capabilities  has  changed  from  100%  solutions,  to  something  less  than  100%  if 
justified  by  reductions  in  development/deployment  cycle  time. 

likely  to  incur  substantial  additional  costs  to  operate  and  maintain  legacy  systems*,.  ”  The 
cost  to  maintain  legacy  systems  must  be  presumed  to  remain  the  same  until  legacy  systems  that 
are  to  be  replaced  by  new  applications  are  eliminated.  For  some  ICP  operations,  only  a  fraction 
of  the  legacy  systems  will  be  replaced.  It  must  be  pointed  out,  however,  that  for  the  past  several 
years  a  bare  bones  approach  to  legacy  maintenance  has  been  sustained.  The  sooner  the 
framework  for  constructing  replacement  applications  for  legacy  systems  is  put  in  place,  the 
sooner  legacy  maintenance  costs  will  be  substantially  reduced.  Imposing  a  delay  on  the 
construction  of  that  framework,  as  recommended  by  the  GAO,  will  only  serve  to  prolong  the 
period  of  legacy  system  maintenance  costs. 

The  GAO  recommendation  is  presumed  to  be  based  on  the  assumption  that  an  alternative  to  the 
current  strategy  exists,  to  wit,  that  the  previous  strategy  is  still  viable.  It  is  not.  The  core  of  the 
materiel  management  ftmctionalities  are  stock  control  and  requirements  computation.  The 
Military  Services  and  Defense  Agencies,  having  reviewed  in  detail  over  the  past  several  years  the 
migration  systems  planned  to  serve  those  two  primary  functions,  have  indicated  that  the  selected 
systems  would  not  bring  sufficient  benefits  to  their  respective  operations  to  justify  the  cost  and 
effort  remaining  to  transition  from  the  current  legacy  systems  performing  those  functions. 

Certain  byproducts  of  the  stock  control  and  requirements  computation  migration  development 
effort  are  being  selectively  deployed  based  on  Service/Agency  desires.  The  Stock  Control 
System  was  deployed  to  the  Marine  Corps  to  replace  the  most  antiquated  of  the  legacy  stock 
control  processes  and  the  Math  Models  algorithms  developed  as  part  of  the  requirements 
computation  process  are  being  selectively  deployed  based  on  customer  desires. 
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The  focus  of  the  new  strategy  is  based  on  the  desire  to  take  advantage  of  changes  in  technology 
and  the  increased  availability  of  commercial  software  products  capable  of  supporting  DoD 
business  practices.  Both  aspects  of  this  focus  will  make  system  maintenance  less  costly  in  the 
long  run  than  the  maintenance  of  the  current  legacy  system  structure  which  is  mainframe-based 
and  which  runs  primarily  unique  government-developed  and  maintained  software.  The 
concentration  on  commercial  technology  and  software  products  will  vastly  increase  the 
modularity  of  the  logistics  business  structure  thereby  reducing  management,  maintenance,  and 
other  sustainment  costs. 

“...is  not  affording  its  own  decisionmakers  an  opportunity  to  thoroughly  review  the  new 
program  before  proceeding  with  deployments.,,**  The  Department  emphatically  non  concurs 
with  this  conclusion.  On  May  13,  1996,  the  MAISRC  decided  that  prior  to  MAISRC  approval  to 
field  these  applications,  the  program  manager  will  prepare  acquisition  and  test  and  evaluation 
strategies.  The  key  elements  of  these  strategies  were  provided  to  the  Air  Force  Program 
Executive  Officer  on  June  24, 1996  for  coordination.  Approval,  when  received  from  the  OIPT 
leader,  the  Director,  DTSE&E,  and  the  Director,  DOT&E,  will  constitute  authorization  to  field 
migration  applications  to  the  initial  test  sites. 

As  noted  above,  a  new  Logistics  Information  Board  has  been  constituted  to  manage  the  logistics 
business  information  process.  This  Board  meets  on  a  monthly  basis  and  its  highest  level  issues 
are  reviewed,  also  monthly,  by  the  senior  logisticians  in  the  Department,  This  same  management 
review  process  will  be  used  to  finalize  the  new  logistics  business  strategy  and  schedule  of 
actions.  The  individuals  involved  in  this  management  process  are  the  Department/Military 
decision  makers  in  logistics.  In  addition,  senior  representatives  of  the  Defense  Information 
Systems  Agency  (DIS  A)  actively  participate  in  this  process  ensuring  that  all  relevant 
technological  actions  are  consistent  with  overall  Department  information  systems  policies  and 
practices  and  are  appropriate  for  the  applications  involved.  It  must  be  emphasized  that  this  level 
of  management  of  the  process  has  been  instituted  because  the  Department  perceived  many  of  the 
same  problems  noted  by  the  GAO  in  its  draft  report.  An  additional  layer  of  review,  compotmded 
by  a  work  stoppage,  as  recommended  would  add  little,  if  anything,  to  the  degree  of  management 
oversight  and  would  delay  progress. 
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1.  The  wording  in  the  paragraph  cited  has  been  modified  but  addresses  the 
same  issue. 

2.  Defense  states  that  it  is  documenting  a  revised  logistics  business 
systems  strategy  at  the  same  time  that  the  first  aspects  of  the  strategy  are 
being  executed.  The  revised  strategy  is  to  be  based  on  the  new  common 
operating  environment  (coe)  approach  for  building  interoperable  systems, 
a  collection  of  reusable  software  components,  a  software  infrastructure 
for  supporting  mission  area  applications,  and  a  set  of  guidelines  and 
standards.  The  standards  will  specify  how  to  reuse  existing  software  and 
how  to  build  new  software  to  facilitate  system  interoperability.  We  are 
currently  revie-wing  the  coe.  While  it  may  address  technical  infrastructure 
problems  associated  with  logistics  business  systems,  we  believe  that 
Defense  must  still  incorporate  into  the  new  strategy  the  essential 
ingredients  that  ensure  sound  decisionmaking  as  we  recommended  in  our 
report;  conducting  required  economic  and  technical  risk  analyses  and 
providing  adequate  oversight  for  the  systems  currently  being  deployed.  In 
doing  so,  Defense  can  ensure  that  the  systems  being  deployed  now  will  be 
compatible  with  systems  or  work  processes  developed  under  the  common 
operating  environment  approach.  Further,  it  can  ensure  that  sound 
business  decisions  are  being  made  as  it  finalizes  the  new  approach. 

Additionally,  while  the  military  services  and  Defense  agencies  have 
requested  materiel  management  deployments,  because  the  systems  have 
not  been  fully  tested,  they  may  not  be  an  improvement  over  existing  legacy 
systems.  In  addition,  in  the  near-term,  these  systems  will  require  the 
services  and  agencies  to  continue  to  maintain  their  legacy  systems.  We 
believe  that  before  these  systems  are  deployed  to  the  services  and  the 
agencies.  Defense  needs  to  ensure  that  testing  is  sufficient.  This  should 
help  reduce  the  risks  of  software  problems  siufacing  in  later  phases  of  the 
system  lifecycle. 

3.  As  stated  in  our  report,  the  economic  analysis  completed  in  1996  was 
not  reflective  of  the  new  approach  to  materiel  management  systems.  An 
analysis  for  the  new  strategy  may  well  have  identified  the  additional  risks 
that  have  not  been  addressed,  such  as  those  associated  with  maintaining 
the  legacy  systems  that  will  now  remain  under  the  new  strategy.  In 
addition,  developing  an  economic  aiialysis  after  deployments  have  begun 
wiU  not  give  Defense  decisionmakers  an  opportunity  to  ensure  that  good 
business  decisions  are  being  made  before  funds  are  committed.  Both  itmra 
and  the  Office  of  Management  and  Budget’s  November  1995  guide  for 
evaluating  information  technology  investments  call  for  such  analyses  prior 
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to  making  information  technology  investments  as  it  allows  senior 
managers  to  examine  trade-offs  among  competing  proposals  and  to  ensure 
that  each  project  is  cost-effective  and  beneficial. 

4.  We  support  Defense’s  efforts  to  communicate  its  intentions  for  materiel 
management  systems  with  management  across  the  services  and  within 
logistics  operations.  Obtaining  support  of  decisionmakers  is  critical  to  the 
success  of  the  new  strategy.  However,  because  Defense  has  not  complied 
with  its  own  regulations  for  ensuring  sound  decisionmaking,  these 
managers  still  do  not  have  the  information  necessary  for  making  informed 
decisions.  FYirther,  the  Logistics  Information  Board,  which  Defense  has 
established  to  review  execution  of  the  new  strategy  is  no  substitute  for  a 
full  MAiSRC  review.  For  example,  while  the  Logistics  Information  Board 
consists  of  members  who  participate  in  logistics  business  operations, 
MAISRC  comprises  high-ranking  officials  separate  from  logistics  business 
who  have  a  more  independent  perspective  in  reviewing  the  strategy. 
Further,  while  the  Logistics  Information  Board  can  play  an  effective  role  in 
development  and  implementation  of  the  strategy,  maisrc  plays  a  vital  role 
in  developing  and  rigorously  verifying  the  cost-benefit  and  alternative 
analyses  that  are  fundamental  to  making  investment  decisions. 

6.  Defense  contends  that  the  systems  it  is  developing  will  be  flexible 
enough  to  support  private  contractors  assuming  responsibility  for  materiel 
management  operations  if  privatization  is  pursued.  This  contention, 
however,  presumes  that  contractors  will  want  to  use  these  systems  rather 
than  acquire  their  own  systems  and  that  these  systems  will  support  new 
work  processes  adopted  by  contractors.  Further,  because  most  of  the 
migratory  systems  being  deployed  are  based  on  an  out-of-date  system 
architectures,  there  is  no  assurance  they  will  facilitate  interoperability 
between  the  legacy  and  cots  environments,  as  Defense  contends. 

6.  The  testing  strategy  dod  describes  is  a  common  practice  in  the 
commercial  world.  However,  it  should  be  noted  that  this  practice  is 
intended  to  mitigate  risk  and  reduce  costs  prior  to  full  production  of  the 
system.  In  dod’s  case,  the  strategy  is  being  employed  during  full 
production  of  the  system.  Additionally,  under  dod’s  approach,  the  services 
will  be  required  in  many  cases  to  continue  spending  operational  funds  on 
their  legacy  systems  to  make  up  for  the  lack  of  full  functionality  in  the 
fielded  new  systems.  Further,  Defense  historically  has  encoimtered 
significant  cost  increases  to  software-intensive  systems  as  a  result  of 
fielding  them  before  they  are  adequately  tested.  As  our  report  discusses, 
these  problems  were  especially  evident  with  the  Stock  Control  System.  In 
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its  comments,  Defense  did  not  dispute  that  the  Marine  Corps  has  had  to 
make  extensive  and  costly  changes  to  the  system  chiefly  because  the 
application  was  delivered  basically  untested  and  with  very  limited 
functionality.  Finally,  we  do  not  beUeve  it  is  appropriate  to  deploy  these 
systems  until  the  testing  strategy  is  approved  by  the  full  maisec. 

7.  We  disagree  with  Defense’s  contention  that  the  costs  to  maintain  legacy 
systems  will  remain  the  same.  Since  a  bare  bones  approach  to  legacy 
maintenance  has  been  sustained  for  the  past  several  years,  we  believe  that 
these  systems  will  require  more  maintenance  as  they  get  older. 
Additionally,  we  believe  the  maintenance  costs  for  legacy  systems  will 
increase  since  fewer  systems  will  be  terminated  imder  the  new  strategy 
than  anticipated  under  the  original  strategy.  As  discussed  in  our  report, 
these  remaining  systems  will  also  require  costly  interfaces  with  new 
systems.  Further,  we  are  not  recommending  that  Defense  delay  moving 
forward  with  a  new  logistics  business  systems  strategy.  Rather,  we  are 
recommending  that  Defense  delay  continuing  to  implement  pieces  of  an 
admittedly  failed  migration  system  strategy  until  it  can  be  assured  that  the 
systems  it  wants  to  deploy  are  good  investments. 

8.  We  agree  that  the  old  strategy  is  not  viable.  However,  we  disagree  that 
there  are  no  alternatives  other  than  the  current  strategy.  Our  report,  in 
fact,  discusses  alternatives  to  the  new  strategy  currently  being  considered 
by  Defense,  such  as  privatizing  materiel  management  functions.  Also,  by 
conducting  required  economic  analyses.  Defense  would  be  able  to  fully 
identify  the  available  alternatives  and  consider  their  associated  costs, 
benefits  and  risks. 

9.  Until  Defense  completes  the  documentation  associated  with  its  own 
oversi^t  process — ^which  includes  a  complete  definition  of  the  new 
strategy;  an  analysis  of  costs,  benefits,  and  alternatives;  and  a  test 

plan — ^its  decisionmakers  will  not  have  assurance  that  they  are  choosing 
the  best  system  solutions.  Further,  the  contention  that  the  full  maisec 
review  of  major  information  system  investments  merely  adds  “another 
level  of  review”  goes  against  Defense’s  original  intention  in  implementing 
this  process;  ensuring  that  the  essential  ingredients  to  making  sound 
business  decisions  are  incorporated  into  all  major  technology  investment 
decisions,  and  that  senior  managers  are  making  the  final  decisions  and 
held  accountable  for  them. 
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Central  Secondary  Item  Stratification  (csis):  Stratifies  the  requirements 
computed  in  the  other  systems  across  financial  programs  and  is  the  basis 
for  budgeting  and  funding  allocations. 

Configuration  Management  Information  System  (cmis):  Provides 
configuration  identification,  configuration  status  accoxmting,  electronic 
change  control,  and  configuration  audits. 

Deficiency  Reporting  System  (drs):  Collects,  processes,  and  stores  quality 
deficiency  and  discrepancy  data  on  weapon  systems  and  equipment. 

Initial  Requirements  Determination/Readiness  Based  Sparing  (ird/rbs): 
Computes  initial  spare  requirements  for  new  systems  and  computes 
requirements  based  on  readiness  scenarios. 

Maintenance  Planning  &  Execution  (mp&e):  Manages  repair  requirements 
and  monitors  the  performance  of  maintenance  facilities. 

Product  Definition  Support  System  (pdss):  Creates  and  moves  a  complete 
requirements  package  from  the  requirements  determination  system  to  the 
contracting  system. 

Provisioiung  Cataloging  Technical  Support  System  (pctss):  Supports  the 
selection  of  items  for  new  end  items/weapon  systems,  obtains  and 
maintains  national  stock  numbers  and  associated  data. 

Requirements  Computation  System  (rcs):  Provides  demand-based 
requirements  computations  for  recoverable  and  consumable  items. 

Stock  Control  System  (scs):  Provides  asset  visibility  through  requisition 
processing,  receipt  processing,  and  inventory  processing. 


Page  37 


GAO/AIMI>-96-109  Defense  IRM 


Appendix  IV 


New  Strategy  Deployments 


Tables  IV.  1  and  IV.2  below  show  the  deployment  schedule  for  fiscal  years 
1996  and  1997.  Four  system  applications — the  Configuration  Management 
Information  System,  Deficiency  Reporting  System,  Initial  Requirements 
Determination/Readiness  Based  Sparing,  and  the  Product  Definition 
Support  System — ^will  be  deployed  in  fiscal  year  1996  at  an  estimated  cost 
of  $3.1  million  at  selected  sites  across  three  of  the  five  services  based 
primarily  on  need.  These  systems  will  be  delivered  to  the  user 
organizations  by  jlsc  “as  is,”  that  is,  with  limited  functionality  and  system 
testing.  According  to  the  mmss  program  manager,  the  number  of  actual 
deployments  for  both  fiscal  year  1996  and  1997  will  be  contingent  on 
available  funding.  In  April  1996,  the  materiel  management  program 
manager  reported  that  the  revised  fiscal  year  1997  schedule  is  likely  too 
ambitious  given  the  funding  projected  to  be  available. 


Table  IV.1:  Deployment  Schedule  for 
Fiscal  Year  1996 


Service 

Configuration 

Management 

Information 

System 

Deficiency 

Reporting 

System 

Initial 
Requirements 
Determination/ 
Readiness 
Based  Sparing 

Product 
Definition 
Support  System 

Air  Force 

0 

0 

0 

0 

Army 

1 

4a 

0 

1 

DLA 

0 

0 

0 

0 

Marine 

Corps 

1 

0 

0 

0 

Navy 

2a 

0 

1 

0 

Total 

4 

4 

1 

1 

Note:  Five  Air  Force  sites  and  one  Marine  Corps  site  will  receive  a  Cobal  software  upgrade  of  the 
Stock  Control  System  in  fiscal  year  1996. 


“Involves  client  and  remote  connectivity  only. 
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Table  IV.2:  Deployment  Schedule  for  Fiscal  Year  1997 

Service 

Configuration 

Management 

Information 

System 

Deficiency 

Reporting 

System 

Product 

Definition 

Support 

System 

Provisioning 
Cataloging 
Technical 
Support  System 

Maintenance 
Planning  and 
Execution 

Initial 
Requirements 
Determination/ 
Readiness 
Based  Sparing 

Air  Force 

2 

3 

0 

0 

2 

0 

Army 

0 

2 

4 

1 

0 

0 

DLA 

1 

2 

4 

1 

0 

0 

Marine  Corps 

1 

0 

0 

0 

1 

0 

Navy 

0 

0 

0 

1 

1 

1 

Total 

4 

7 

8 

3 

4 

1 

Note:  Five  Air  Force  and  one  Marine  Corps  sites  will  receive  a  Stock  Control  System  upgrade  to 
correct  reported  deficiencies. 


Defense  hired  a  contractor  to  conduct  site  surveys  at  each  deployment  site 
to  determine  the  physical  plant  and  architectural  requirements,  that  is, 
communication,  electrical,  and  computer  hardware  and  software 
configurations  needed  to  support  applications.  As  of  May  31, 1996,  the 
contractor  had  completed  11  of  the  21  required  site  surveys.  According  to 
JLSC  officials,  to  meet  their  deployment  schedule,  some  applications  will  be 
deployed  in  fiscal  year  1996  even  though  the  site  surveys  may  not  be  done. 
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Georgia 


By  fielding  the  Stock  Control  System  (scs),  jlsc  expected  to  achieve 
immediate  benefits  and  to  demonstrate  major  progress  in  support  of  dod 
objectives.  The  benefits  to  the  Marine  Corps — ^the  first  service  to  receive 
the  system — ^included  replacing  an  outdated  system  and  moving  from  a 
batch  system  to  on-line  processing.  The  monetary  benefits  were  expected 
to  exceed  $56.7  million  with  an  implementation  cost  of  $27.2  million.^ 
MCLB-Albany  was  selected  as  the  scs  production  site  in  August  1993,  and 
the  system  was  declared  operational  in  April  1995.  However,  scs  was  not 
fielded  with  full  functionality. 

In  addition  to  the  challenges  of  learning  a  new  system,  users  continue  to 
experience  problems  with  the  system’s  operability.  They  cited  the 
following  reasons  for  the  difficulties  encoimtered  with  the  scs: 

•  The  majority  of  users  believed  that  the  initial  testing  of  scs  was 
inadequate.  Since  the  system’s  deployment,  users  have  experienced 
problems  that  should  have  been  detected  during  the  system’s  testing. 

•  Most  users  reported  not  receiving  timely  training  prior  to  the  system’s 
deployment.  Given  that  training  was  provided  up  to  8  months  before 
system  implementation,  the  majority  agreed  that  the  training  was  too  early 
for  them  to  retain  the  knowledge  they  needed  to  operate  the  system.  An 
MCLB-Albany  official  stated  that  training  was  conducted  so  far  in  advance 
because  the  deployment  date  was  officially  scheduled  to  be  6  months 
earlier.  Three  of  the  users  stated  that  training  was  ineffective  because 
instructors  were  unfamiliar  with  Marine  Corps  processes  or  too  general  in 
their  presentation.  Recognizing  the  need  for  additional  training, 
MCLB-Albany  conducted  a  refi'esher  class  just  prior  to  the  implementation 
of  scs,  which  some  of  the  users  thought  was  beneficial. 

•  Most  of  the  users  believe  that  the  scs  system  documentation  is  insufficient 
and  several  thought  the  manual  was  useless  and  ever  changing.  One 
inventory  manager  never  even  received  a  user’s  manual.  All  of  the  users 
seek  answers  to  their  problems  by  consulting  with  designated  scs  system 
analysts  or  other  users. 

•  All  of  the  users  stated  that  the  scs  no  longer  allows  them  to  perform 
certain  job  related  tasks.  In  specific  instances,  some  of  the  users  reported 
going  to  a  separate,  manual  source  to  complete  these  tasks.  An 
MCLB-Albany  official  anticipates  resolving  this  problem  with  later  versions 
of  system. 

.  Most  of  the  users  have  experienced  problems  in  accessing  scs,  which  has 
been  vmavailable  for  periods  of  time  ranging  from  a  few  minutes  to  several 


^As  of  March  1996,  MCLB-Albany  has  received  $14,063,474  to  implement  SCS  and  has  spent  $14,040,515, 
with  the  majority  of  this  money  going  to  the  system’s  development  contractor. 
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days.  Given  that  users  spend  as  much  as  90  percent  of  their  day  in  scs,  this 
problem  could  inhibit  their  ability  to  do  their  jobs.  MCLB-Albany  officials 
stated  that  the  inaccessibility  of  scs  is  often  due  to  problems  with  their 
local  area  network  rather  than  problems  with  the  scs. 

An  official  with  MCLB-Albany’s  Defense  Accounting  Office  reported  that 
financial  data  is  not  able  to  pass  directly  from  the  scs  to  the  accounting 
system. 

All  nine  inventory  managers  interviewed  liked  that  scs  gave  them 
immediate,  on-line  processing  of  information.  The  legacy  system  employed 
batch  processing,  making  users  wait  until  the  day  after  they  input 
information  to  receive  the  results. 

Needing  a  success  story,  jlsc  set  unrealistic  milestone  dates  and  pushed 
the  system  through  testing  and  onto  the  users.  One  official  told  jlsc  that  its 
scheduling  of  testing  actually  extended  time  frames  and  resulted  in  a  loss 
of  confidence  from  the  users.  This  official  and  a  lessons-leamed  report 
emphasized  that  the  manner  in  which  the  system  tests  were  conducted  has 
exacerbated  the  problems  with  scs.  They  cited  that 

some  problems  were  not  identified  in  testing; 

the  tests  only  addressed  one  area  of  the  system  at  a  time,  as  if  in  a  vacuum; 
the  tests  were  conducted  on  a  different  operating  environment  from  the 
Marine  Corps’; 

some  areas  of  scs  were  not  tested;  and 

insufficient  testing  at  contractor  facility  led  to  additional  on-site  testing  to 
correct  problems. 

Albany  has  devoted  many  resources  to  implement  interim  solutions  to 
resolve  interface  and  operating  environment  problems.  Analysts  have 
resorted  to  these  measures  because  scs  has  on-line  processing  capabilities, 
while  the  legacy  systems  use  batch  processing.  Additionally,  scs  uses  a 
different  operating  environment  from  the  legacy  systems.  Even  with  these 
interim  solutions,  MCLB-Albany  cannot  ensure  that  data  is  carried  from  one 
operating  environment  to  the  next.  Additionally,  because  MCLB-Albany  has 
scs  and  legacy  systems  operating  in  tandem,  it  spends  more  time  and 
money  maintaining  and  reconciling  the  systems  and  to  purify  and  convert 
data.  Although  MCLB-Albany  officials  acknowledge  that  a  cost  exists  for  all 
of  these  interim  solutions,  they  were  imable  to  determine  the  cost  of 
addressing  interface  problems. 
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According  to  Program  Managers,  jlsc  ceased  to  provide  feedback  to 
MCLB-Albany  on  scs  monthly  activity  reports.  At  the  beginning  of  the  effort, 
JLSC  required  status  reports  from  mclb- Albany,  but  they  later  directed 
Albany  to  discontinue  these  reports.  Considering  that  scs  still  has 
unresolved  problems,  jlsc’s  instructions  to  cease  the  flow  of  progress 
reports  is  puzzling.  MCLB-Albany  officials  later  informed  us  that  informal 
communication  with  jlsc  has  subsequently  improved  with  the  new 
direction  of  mmss;  however,  they  still  receive  no  feedback  on  their  progress 
reports.  MCLB-Albany  continues  to  generate  the  status  reports  for  their  own 
benefit. 

Although  the  users  continue  to  experience  problems  using  scs,  program 
management  at  MCLB-Albany  believes  that  scs  will  be  its  asset  management 
system  into  the  future.  MCLB-Albany  is  scheduled  to  receive  an  update  to 
scs  in  1996,  which  includes  an  upgrade  to  a  modernized  language  because 
the  older  version  will  no  longer  be  supported  by  the  commercial  market. 
The  new  language  is  expected  to  alleviate  incompatibility  problems  in  the 
operating  systems.  The  Marines  consider  this  update  a  top  priority. 
Althou^  officials  at  MCLB-Albany  continue  to  request  the  functions  that 
they  did  not  get  when  scs  was  deployed,  jlsc  has  not  responded.  These 
missing  functions  are  now  under  initiatives  of  other  dod  agencies. 
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